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@ Electronic video system with simultaneous real-time processing. 

(5?) The system comprises a store 1 1 having sev- o 
era! disc drives for storing multiple video frames 2 < 

in random access order. The store 1 1 also has at | a 

least two ports 15, 16 each capable of transfer- 
ring data relating to a respective video clip at 
video rate. The system further comprises pro- 
cessors 27, 28 for processing frame data as- 
sociated with at least two video clips and a 
monitor for displaying selected ones of said 
multiple video frames. In a preview mode of 
operation video clips can be read from the store 
11, combined and displayed on the monitor at 
video rate, and changes can be made to the 
dips and the manner in which they are com- 
bined without committing the combination to 
the store. 
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The invention relates to an electronic video proc- 
essing system. The invention also relates to an elec- 
tronic video store. 

A video graphics system is described in our Brit- 
ish Patent No. 2,089,625 and corresponding United 
States Patent No. 4,514,818, the teachings of which 
are incorporated herein by reference. This system in- 
cludes a touch tablet and stylus combination for gen- 
erating position signals designating the position of the 
stylus relative to the touch tablet. The user or opera- 
tive paints or draws (hereinafter referred to simply as 
"paints") by selecting a colour and a so called brush 
distribution from a range of predefined colours and 
distribution functions. Movement of the stylus relative 
to the touch tablet causes the generation of a series 
of position signals. 

In equipment according to our British patent no. 
2,089,625, as currently manufactured and sold, addi- 
tional means are provided which allow the system to 
be configured to perform picture composition in addi- 
tion to painting. Some configurations which allow 
composition are described in our British patent no. 
2,113,950 and in corresponding US patent no. 
4,602,286 the teachings of which are also incorporat- 
ed herein by reference. In such configuration, a store 
is provided having the capacity to store two indepen- 
dent pictures, termed herein as the foreground image 
and the background image respectively, and a further 
image which is termed the control image or stencil. 
The control image, like the foreground and back- 
ground images, can be created by the artist, using the 
system in a painting mode. The control image usually 
comprises areas of pixels having a maximum value 
which represents opacity and a minimum value which 
represents transparency and is created so that in ef- 
fect it masks a selected part of the foreground image 
while exposing the remainder. The control image is 
defined by eight bits and during creation can be dis- 
played as a single colour superimposed on the fore- 
ground picture. The pixels of the foreground image 
and the control image are read in parallel from the re- 
spective parts of the store and applied to a linear in- 
terpolating circuit, which is rendered operational dur- 
ing an image preview mode and during an image com- 
position mode. The pixels in the background image 
are also read to the interpolating circuit concomitantly 
with the reading of the foreground image pixels and 
the control image pixels. 

A manipulator under stylus control or the control 
of other user operable means, changes the order of 
reading the foreground pixels and control pixels rela- 
tive to the background pixels so as to effect a spatial 
transformation of the foreground and stencil relative 
to the background. Whilst the system is in preview 
mode, the artist can manipulate the stylus to effect a 
series of desired transformations such as zoom, pan, 
scroll, rotate, and change perspective. During a frame 
period the foreground image pixels in transformed or- 



der and the background image pixels are applied to 
an image input of the interpolating circuit whilst the 
control image is applied to a control input thereof to 
act as a pixel- by-pixel interpolating coefficient. The 

5 output of the interpolating circuit is applied to the pic- 
ture monitor and displayed, but without affecting any 
of the pictures as stored. The interpolation is of the 
form KB+(1-K)F, where K is an interpolation constant 
defined by the control image, M is the background 

10 picture data and F is the foreground picture data. The 
interpolation has the effect of causing the display to 
represent the foreground image where the control im- 
age data K=0 ( and to represent the background image 
where the control image data K-1, and to represent a 

is combination foreground and background where K has 
a value between 0 and 1 , thereby simulating the past- 
ing or overlaying of the part in the foreground image 
on the background. Using the stylus to vary the spa- 
tial transformation, the artist can try various positions 

20 of the insert onto the background image from the 
foreground image until he perceives a desired trans- 
formation. Thereupon the artist, by producing a pre- 
determined "stick" command, switches the system to 
the composition mode. The foreground image pixels 

25 and background image pixels are then read to the in- 
terpolating circuit and interpolated under control of 
the control image pixels, the foreground image and 
control image pixels being read in the transformation 
order prescribed in the last try during the preview 

30 mode. 

The features and facilities provided by the above 
discussed systems can be applied to television post 
production in the form of an editing system in which 
a first or foreground video clip is inserted into a sec- 

35 ond background video clip. Such editing systems 
nowadays have the ability to capture and retouch a 
frame of live video, thereby enabling dynamic graph- 
ics to be produced. 

Hitherto known editing systems vary from manu- 

40 facturer to manufacturer but generally comprise the 
features shown in Figure 1 of the accompanying 
drawings. As shown in Figure 1 of the accompanying 
drawings a known editing system 1 comprises two 
sources 2 and 3, which may for example be video tape 

45 recorders (VTRs), whose outputs are connected to a 
vision mixer 4 which together with the two sources 2 
and 3 are controlled by an edit controller 5. The sys- 
tem 1 further comprises a graphics unit 6 such as the 
abovementioned system disclosed in our British pa- 

so tent 2,089,625 and an effects unit 7 by which spatial 
transformations, such as those disclosed in our 
abovementioned British Patent 2, 1 1 3,950, may be ap- 
plied to video from one or both of the sources 2 and 
3. 

55 The system 1 can be used to apply graphics and 
effects to video from one of the sources, for example 
the source 2. In this mode of operation, a video clip 
is output a frame at a time from the source 2 to the 
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effects/graphics unit 6, 7 where desired modifications 
are effected by the user to the frame. Once the frame 
has been modified it is transferred back to the source 
2 and the next frame in the clip is output to the unit 
6, 7. During modification the frame is output for dis- s 
play on a monitor 8 so that the user can see the result 
of his modifications as they are made to the frame. 

The system 1 can also be used to combine video 
clips from the two sources 2 and 3, which clips may 
be modified as described hereinabove prior to being w 
combined. One way in which the clips may be com- 
bined is simply to join or splice one clip to the end of 
the other or to splice one clip or a portion of that clip 
into the middle of the other clip. In this mode of op- 
eration the edit controller 5 is made responsive to an 15 
edit decision list which identifies the location of each 
clip in the two sources by for example its start frame 
and end frame, the number of frames in the clip and 
where appropriate the frames between which a splice 
is to be made. A more sophisticated operation which 20 
may also be performed by the system 1 is that of com- 
bining two clips by dissolving between the two clips. 
In a dissolving operation the edit decision list includes 
data identifying the number of frames in a dissolve. In 
response thereto the edit controller 5 controls the 25 
mixer 4 to combine corresponding frames in the two 
clips, for example in a weighted summing of the frame 
data. During this mode of operation frames from the 
source 2 may be modified by the effects unit 7 before 
being passed to the vision mixer 4. 30 

In all of the above discussed modes of operation 
the resulting video output from the mixer 4 may simply 
be displayed on "the monitor 8 or it may be stored to 
enable further work to be carried out subsequently. 
In the case where it is stored for subsequent work a 35 
further VTR (not shown) will be required to record the 
video output. A further VTR is necessary since both 
of the source VTRs 2, 3 will be engaged in playing out 
the two video clips and therefore will not be available 
for storing new frames as they are created. 40 

The abovedescribed system 1 therefore may in- 
clude a third VTR (not shown) which is used to record 
the combined clip output from the mixer 4. Such a 
system is known as a three machine edit suite, since 
it comprises three VTRs. A further VTR may be added 45 
to the system as a third source (not shown) to form a 
so-called four machine edit suite. During editing the 
third source is used to supply a moving black and 
white key, known as a "matte reel". The key is related 
to the image in a clip supplied by one of the sources so 
2, 3 and is used in the mixer 4 to key that clip over the 
video supplied by the other of the sources 2, 3. 
Frames from the matte reel may be passed together 
with corresponding frames from the source 2 to the 
effects unit for combination with respective frames 55 
from the source 3 by the vision mixer 4. 

One drawback with the abovedescribed three 
and four machine edit suites is that often more than 



one operation is required to produce a desired result 
and this restricts artistic use of the system. Further- 
more, it is impossible to create graphics dynamically, 
ie as the video clips are run and this prevents subtle 
integration of the graphics with the video resulting in 
a discernable difference between the live video and 
the off-line graphics. As a result, it is difficult to create 
first time around exactly what was intended and the 
creation process is therefore one of repeatedly work- 
ing and reworking a piece until the desired result is 
achieved. 

Another drawback is related to the use of VTRs 
to supply and record video clips. VTRs are sequential 
access devices and cannot simultaneously playback 
and record different video clips. This means that a 
separate VTR is required as a source for each video 
clip to be worked on and at least one further VTR is 
required to record the video output from the mixer. 

We manufacture and sell an editing system under 
the trade mark "HARRY". In our HARRY system 
frames forming a video clip are digitally encoded and 
stored in disk stores. As is disclosed in our US Patent 
No. 4,686,106, the teachings of which are incorporat- 
ed herein by reference, the video clips are read out of 
the disc stores in parallel. Whilst this arrangement 
provides greater flexibility by effectively allowing ran- 
dom access to the frames that constitute a clip, it nev- 
ertheless requires separate disc stores for each of 
the video clips. 

The present invention in one of its aspects aims 
to overcome the above and associated problems by 
providing an electronic video store comprising storing 
means for storing digital data representing multiple 
video frames and access means for providing random 
access input and output of video frames to and from 
the storing means and including at least two ports 
connected to the storing means and each capable of 
transferring data to or from the storing means relating 
to a respective video frame at video rate. 

Since the access means enables random access 
to the stored video frames, one port might for exam- 
ple be used to read frames relating to one clip from the 
store whilst the other port is used simultaneously to 
read frames relating to a second clip from the store, 
thereby enabling simultaneous real-time processing 
and display of the two clips by the system. 

The combination of a random access store to- 
gether with at least two input/output ports provides 
for a system of far greater flexibility than has hitherto 
been possible. 

In the practice of the invention each of the ports 
has sufficient bandwidth to enable at least one clip to 
be transmitted at video rate. Since there are at least 
two ports it is therefore possible to convey simultane- 
ously frames relating to at least two video clips from 
the store for concurrent processing and/or display of 
the images represented thereby. Thus, the two ports 
enable a dissolve between two video clips to be made 
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as the data is required, without the need to store the 
dissolve clip during its creation. This facilitates editing 
by allowing a dissolve to be previewed before the data 
therefor is committed to memory. 

In another aspect the invention provides an elec- 
tronic video processing system for combining a plur- 
ality video clips, the system comprising storing 
means for storing a multiplicity of video frames and 
access means for providing random access transfer 
to and from the storing means of said video frames 
comprising said plural video clips and including at 
least two ports connected to the storing means and 
each capable of transferring data to or from the stor- 
ing means at video rate, the system further compris- 
ing a processor responsive to user operable input 
means for processing data from the storing means 
representing user selected frames of said two video 
clips to produce a combined video clip, and a monitor 
for simultaneously displaying at least some of said 
multiplicity video frames. 

With the intention of facilitating editing the pres- 
ent invention aims to provide for greater interaction 
with the user by enabling several of functions to be 
selected simply by movement of a cursor on the dis- 
play and manipulation of a stylus on a touch tablet. 

The above and further features of the invention 
are set forth with particularity in the appended claims 
and together with advantages thereof will become 
clearer from consideration of the following detailed 
description of an exemplary embodiment of the inven- 
tion given with reference to the accompanying draw- 
ings. 

In the drawings: 

Figure 1 is a schematic diagram of a prior art sys- 
tem as already described hereinabove; 
Figure 2 is a schematic diagram of a system em- 
bodying the invention; 

Figure 3 is a schematic diagram of a video store; 
and 

Figure 4 is an exemplary display of video frames 

generated by the system of Figure 2. 

Referring now to Figure 2 of the accompanying 
drawings there is shown an electronic video process- 
ing system 10 in which video clip data is stored in a 
video store 11. The video data is input to the video 
store 11 via an input selector 12 which includes both 
an analog input 13 and a digital input 14 to enable data 
to be input in either analog or digital format from an 
off-line bulk storage device or library (not shown) 
such as a VTR. The video clip data is stored in digital 
form in the video store 11 and the input selector 
therefore comprises a suitable converter for convert- 
ing the incoming clip data into the required format. 
The input selector 12 is connected to the video store 
11 via two bidirectional data paths 15, 16 each capa- 
ble of conveying a full frame of data at at least video 
rate to and from the video store 11 . 

The video store 11 is shown in greater detail in 



Figure 3 of the accompanying drawings as compris- 
ing a converter 17 which interfaces the two bidirec- 
tional paths 15, 16 via two buffers 18, 19 to at least 
two disc packs 20, 21. Each disc pack comprises for 

5 example two Fujitsu 2380 disc drive devices in paral- 
lel. Each pack 20, 21 requires two disc drives in order 
to achieve the necessary bandwidth since each disc 
is only able to receive or output data at approximately 
half broadcast standard video rate. For each path 1 5 t 

10 16 to be able to convey data at broadcast standard 
the paths each require a bandwidth of approximately 
21 M bytes per second. Each Fujitsu disc drive has a 
bandwidth of approximately 16 M bytes per second. 
The four disc drives therefore have more bandwidth 

is capacity than is actually required. This extra capacity 
provides time for the discs to locate data for the next 
frame to be output during the next frame period or to 
locate an area for storing the next frame during the 
next frame period. Thus by providing two disc drives 

20 in parallel each disc pack 20, 21 is able to receive or 
output data at full broadcast standard video rate (ie 25 
or 30 frames per second) and the video store 11 is 
therefore able to output in a frame period or store all 
the data for two full frames of video. 

25 The buffers 18, 19 cooperate with the compres- 
sor 17 to distribute data from the two bidirectional 
paths 15, 16 to the disc packs 20,21 when data is be- 
ing written to the video store 11 , and to combine data 
from the disc packs 20, 21 to the bidirectional paths 

30 15, 16 when data is being read from the video store. 
For example, when two frames of data are read simul- 
taneously from the store 11, the data relating to the 
two frames is read in one frame period from the disc 
packs 20, 21 via the buffers 18, 19 to the compressor 

35 17 where the data is reformed as two separate 
frames which are output separately, one on each line 
15, 16 during the next frame period. Similarly, two in- 
coming frames of data received by the compressor 1 7 
from paths 15 and 16 in one frame period are distrib- 

40 uted via the buffers 18, 19 to the disc drives in the 
disc packs 20, 21 during the next frame period. In or- 
der to facilitate the random access transfer of frame 
data between the store 11 and the rest of the system 
10, the converter 17 and/or each buffer 18, 19 in- 

45 eludes a large cache store (not shown) for accumulat- 
ing several frames of data. This minimises the amount 
of head seeking by the disc drives and applies equally 
to both the reading and writing of data. 

The frame data may be distributed among the 

so disc drives in any convenient order. For example one 
field of the frame may be delivered to buffer 18 and 
from there the chrominance of the field written to one 
disc drive in the pack 20 and the luminance of the 
field written to the other disc drive in the same pack 

55 20. The otherf ield of the frame would be delivered to 
the other disc pack 21 for storage therein in a similar 
manner. Thus, the elements of each frame are distrib- 
uted among the four disc drives of the two disc packs 
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20,21. 

It will be appreciated from the foregoing that the 
video store is a truly random access frame store 
which thereby enables frames from different video 
clips to be accessed in a random order for output 
therefrom. The provision of two bidirectional paths 
15,16 enables two frames of data to be simultaneous- 
ly written to or read from the store 11 or for one frame 
to be written to while another is read from the store 
11. This enables a system 10 of greater flexibility to 
be produced than has hitherto been possible. 

Returning to Figure 2, the system 1 0 further com- 
prises a display store 22 which is connected via two 
bidirectional busses 23, 24 to the input selector 12. 
The display store 22 comprises a large scratch pad 
store for storing data relating to several frames of vid- 
eo and a display processor for processing the frame 
data therein to produce respective frames of reduced 
size for concurrent display at different portions of a 
monitor 25, as will be described in greater detail here- 
inafter. The store is able to output one piece of stored 
video simultaneously to several different places on 
the monitor. The display processor comprises two 
separate processing units for producing simultane- 
ously images of reduced size from data supplied via 
the two bidirectional busses 23, 24. A video clip may 
be read out from the video store 11 and written directly 
to the display store 22 at video rate for display on the 
monitor 25 either at full size or at half or quarter size. 
Alternatively video clips may be transferred from a 
bulk storage library (not shown) via the input selector 
12 to the display, store 22. Data from the display store 
22 is read by an -output selector 26 and output to the 
monitor 25 for display. This enables the user to pre- 
view one or more video clips or to identify video clips 
stored off-line in a bulk store library (not shown) for 
transfer to the video store 11. The output selector 26 
also comprises both analog and digital outputs 26a, 
26b, similar to the analog and digital inputs 13, 14 of 
the input selector 12, to enable video clip data to be 
output in either analog or digital format to the bulk 
storage device for example. 

In an editing mode of operation data relating to a 
plurality of video clips is read out from the video store 
11 a frame at a time to the video store 22 where the 
data is processed before being stored so as to enable 
several frames to be displayed simultaneously at dif- 
ferent, overlapping, or shared, portions of the monitor 
25, as will be described in greater detail hereinafter. 
The processed data is output from the display store 
in raster order for display on the monitor 25. The video 
store has associated with it a video processor 27 and 
an effects processor 28. The video processor 27 is ar- 
ranged to perform such operations as generating a 
keying signal, modifying colour, changing texture, and 
the effects processor 28 generates spatial effects 
such as changes of size, position and spin to one or 
more frames of a video clip, which operations and ef- 



fects all per se well known. Frames modified by the 
video processor 27 and effects processor 28 are sup- 
plied or returned to the display store 22 for display on 
the monitor 25. 

5 Preferably the parameters controlling the opera- 

tions executed by the video processor 27 and the ef- 
fects processor 28 are variable from frame to frame. 
So-called key frames may be defined by the user and 
parameters for frames between key frames derived 

10 by interpolation. Alternatively t he parameters control- 
ling the operations of the video processor 27 can be 
fixed so that only parameters controlling the spin, 
position, size, etc operations of the effects processor 
28 are variable. The use of key frames to define se- 
ts lected parameters is per se well known and is dis- 
closed for example in our British Patent Application 
published as No. 2,223,910, the teachings of which, 
are incorporated herein by reference. 

The display store 22 also has associated with it 

20 a graphics processor 29 which enables a user to paint 
into each frame as described in our abovementioned 
British Patent No. 2,089,625, and/or to create a key or 
stencil as described in our abovementioned British 
Patent No. 2,113,950. The graphics processor 29 is 

25 responsive to a user operable input device such as a 
stylus and touch tablet combination 30, as indeed are 
the video processor 27 and the effects processor 28. 

During editing the video processor in the display 
store 22 is arranged to generate from the input video 

30 clips a display that represents an editing environment 
such as the display shown in Figure 4 of the accom- 
panying drawings. As shown in Figure 4 of the draw- 
ings, the display is divided into three distinct working 
areas 31, 32, 33 by three so-called reel bars 34, 35, 

35 36. Each working area 31, 32, 33 has displayed in it 
a small number of frames relating to one or more reels 
or video clips 37 to 44. Of course more than three 
working areas could be displayed if so desired. A high 
definition television monitor for example has a wider 

40 aspect ratio (16x9) than normal broadcast television 
(4 x 3) and would be well suited to the display of four 
or even five reels. 

Each reel bar 34 to 36 includes a marker 45 to 47 
which moves along the reel bar to provide an indica- 

45 tion of the position of the displayed frames in relation 
to the clip as a whole and each displayed frame in- 
cludes at its bottom right hand corner an information 
box, for example 48 to 56, to provide a further indica- 
tion of the positions of the displayed frames in the vid- 

so eo clips. The information in each box may for example 
be the number, ie position, of the frame in the clip or 
it may be time codes generated by a VTR when the 
frames were originally recorded. Each video clip can 
be worked on independently in each working area 31, 

55 32, 33 and can be moved into another of the working 
areas to be combined with another video clip dis- 
played therein, by movement of the stylus on the 
touch tablet as represented by a cursor 57 displayed 
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on the monitor 8 as will be described in greater detail 
hereinafter. 

The cursor 57 is used to control many operations 
performed by the system 10. One function which the 
cursor 57 can be used to control is that of selecting s 
which frames from a selected clip are displayed in the 
working areas 31 to 33. The reel bars 34 to 36 each 
have associated direction control icons 34a to d, 35a 
to e and 36a, to e. The reel bars, the control icons and 
the cursor are all generated by the processor within 10 
the display store 22. Placing the cursor 57 over the 
icon 36a for example and pressing the stylus firmly 
on the touch tablet will cause the clip displayed in 
work area 33 to scroll up whilst selecting icon 36b will 
cause the clip to scroll up to an event, such as a join, is 
previously defined in the clip. Similar operations are 
performed by selection of the other icons as will be 
readily apparent to those possessed of the appropri- 
ate skills. When a scrolling icon is selected data relat- 
ing to the appropriate video frames in the appropriate 20 
clip are read from the video store 1 1 to the video store 
22. The data is processed to reduce the size of each 
frame and the processed data is then stored at loca- 
tions in the store 22 corresponding to the position of 
the reduced frame on the monitor 25. 25 

Video clips to be edited are loaded from the bulk 
store (not shown) via the input selector 12 to the video 
store 11. One of the bidirectional paths eg 15 is used 
to transfer the incoming video clip data to the video 
store 11 and this leaves the other path eg 16 free to 30 
transfer data relating to another video clip, for exam- 
ple a moving key associated with the first mentioned 
video clip. During the loading of clip data in the store 
11 the data is read simultaneously by the display 
store 22 for display of the video on the monitor. 35 
Indeed, the video of both clips can be displayed at the 
same time either as two half size images duly proc- 
essed by the display store 22 on the monitor 25 or as 
full size images using two monitors. Selected frames 
of selected clips stored in the video store 11 can then 40 
be transferred to the video processor 27 and effects 
processor 28 for processing and to the display store 
22 for display on the monitor 25. Since the video store 
22 is effectively a random access frame store, the 
system can be made to display any frame from any 45 
clip or combination of frames and clips in any order 
and at any position on the monitor 25 without the 
need for the user to commit irretrievably to a particu- 
lar displayed combination until such time as he is en- 
tirely happy with the result as displayed on the mon- so 
itor. 

Once the desired clip or clips have been loaded 
into the video store 11 and selected frames therein 
displayed on the monitor 25 the user can edit the 
clips. Cut and insert editing, ie video splicing, is ach- 55 
ieved for example by displaying frames of a first video 
clip to be inserted in say the working area 32, and 
frames of a second video clip, to be cut to receive the 



first dip, in the working area 33. The start of the first 
clip is identified by placing the cursor 57 between the 
first frame (for example frame 58 in Figure 4) and its 
preceding frame and pressing the stylus down firmly 
on the touch tablet and thereafter moving the cursor 
rapidly to the right by corresponding rapid manipula- 
tion of the stylus on the touch tablet. This action is a 
cutting action known as swiping and is interpreted by 
the processor 27 as a command to cut the first video 
clip at the join between frame 58 and the preceding 
frame (indicated at 59 in Figure 4) to form an insert 
video clip (indicated at 42 in Figure 4). 

The image of the insert clip 42 can be attached 
to the cursor 57 by placing the cursor over the first 
frame 58 and pressing the stylus down on the touch 
tablet Thereafter two dimensional movement of the 
cursor 57 on the monitor display results in corre- 
sponding movement of the insert video clip 42 on the 
display. 

The user is therefore free to move the insert vid- 
eo clip 42 anywhere on the display screen and can for 
example move the insert clip 42 into the working area 
33 and position the first frame 58 thereof over a frame 

60 of the second video clip displayed therein. With 
the insert clip 42 so positioned, pressing the stylus 
down firmly on the touch tablet is interpreted by the 
processor 27 as a command to cut the second video 
clip at the join between frame 60 and the preceding 
frame (61 in Figure 4), and to join the insert clip 42 
onto the end of the second clip after frame 61 thereof. 

Once this operation has been performed by the 
processor a new clip is defined as a spliced combin- 
ation of the frames of the second clip up to and includ- 
ing frame 61 followed by alt frames after and including 
frame 58 of the first clip. A similar operation can be 
used to move a clip from one reel to another. If instead 
of placing the first frame 58 of the first clip. over the 
frame 60 of the second clip, the first frame 58 is in- 
stead placed over the join between the frames 60 and 

61 of the second clip, then the whole of the first clip 
will be placed between frames 60 and 61 of the sec- 
ond clip. 

Since the cut and insert operation is performed 
in the video processor 27 it therefore does not result 
in a corresponding change in the frame data in the 
video store 11. Accordingly, data defining frame 59 
and the preceding frames in the first clip and data de- 
fining frame 60 and subsequent frames in the second 
clip remain available for further processing if re- 
quired. This availability of frames is indicated on the 
display by a continuous line 62 which crosses the 
combined clip at the join between the first and second 
clips and connects between a pair of tails 63, 64 run- 
ning parallel to the displayed clip. The length of each 
tail 63, 64 represents the number of available frames 
in the respective first or second clip. Thus the tail 64 
indicates that the first two frames of the first clip do 
not form part of the combined clip and are available 
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for further processing if required, and the tail 63 indi- 
cates that the last three frames of the second clip do 
not form part of the combined clip and are available 
for further processing if required. 

Making frames available for further processing 
can be useful if the operator is not happy with the re- 
sult obtained in the combined clip during preview 
when the clip is run and displayed as a video se- 
quence. In order to facilitate user changes to the com- 
bined clip by changing the position of the join in the 
combined dip, the processor 27 is made responsive 
to manipulation of the stylus and corresponding 
movement of the cursor 57 in the area of the join as 
represented by the cross bar 62. When the cursor is 
placed over the cross bar 62 and the stylus is pressed 
down firmly on the touch tablet subsequent upward 
movement of the cursor 57 on the display screen 
causes the processor to insert in reverse order 
frames from the first clip and to remove in reverse or- 
der frames from the second clip. Thus, in Figure 4 up- 
ward movement of the cursor would cause frame 61 
of the second clip to be replaced by frame 59 of the 
first clip. The length of tail 64 would decrease accord- 
ingly by one frame and the length of tail 63 would in- 
crease accordingly by one frame. Similar downward 
movement of the cursor 57 would result in the oppo- 
site change in frames occurring. Furthermore, hidden 
frames (ie frames represented by the tails 63, 64) in 
either clip can be displayed by placing the cursor 57 
on the tail 63, 64 and swiping upwards along the tail. 
The frames thus revealed can be hidden again by 
placing the cursor 57 on the tail and swiping down- 
wards along the'Tail, 

The system is also arranged to facilitate replica- 
tion of a clip or a portion of a clip. The clip or clip por- 
tion to be replicated is identified by placing the cursor 
over the first frame thereof, pressing the stylus down 
firmly on the touch tablet in order to connect the clip 
to the cursor and then pressing down again with the 
stylus to disconnect the cursor from the clip. The dis- 
play store 22 responds to this manipulation by repli- 
cating the identified frames in the store. The copy clip 
thus created is not displayed separately on the mon- 
itor but is conceptually held or displayed off-screen. 
The system is therefore arranged to enable the copy 
clip to be retrieved by swiping the cursor off one side 
of the screen. This causes the copy clip to be attach- 
ed to the cursor and the copy can then be positioned 
as desired on the screen. 

In addition to splicing two clips together, as dis- 
cussed hereinabove, the system 1 0 is able to dissolve 
between two clips over a selected number of frames. 
In a procedure similar to that already discussed here- 
inabove the first and last frames of two video clips are 
identified and the two clips are spliced together in one 
of the working areas 31 to 33. A dissolve is then set 
over a selected number of frames, by placing the cur- 
sor 57 over the cross bar 62 at the join between the 



two clips and pressing down with the stylus. The sys- 
tem interprets this action as a command to create a 
dissolve. The command is acknowledged by the 
crossbar 62 being displayed in a different colour. A 
5 menu is displayed on the monitor thereby enabling 
the user to enter data about the dissolve including its 
length and whether the dissolve starts at splice point 
as represented by the crossbar 62, finishes at the 
splice point, or includes the splice point. Hidden 
10 frames that contribute to the dissolve are represented 
by respective portions of the tails 63, 64 being dis- 
played in a further different colour. Again, there is no 
actual change in the data stored in the video store 11 
at this stage and the user is free to change the posi- 
ts tion of the join between the two clips as described 
hereinabove. 

In some situations the user may wish to use only 
some of the hidden frames in the two clips, as repre- 
sented by the tails 63, 64, in a dissolve operation. For 
20 example, the user may wish to splice the first and 
second clips together such that the first clip has say 
ten hidden frames after the splice and the second clip 
has say six hidden frames before the splice. The user 
may then decide to dissolve between the two clips 
25 over say ten frames centred on the splice. As such the 
dissolve five of the hidden frames from each of the 
clips. Hidden frames used in the dissolve are indicat- 
ed by the appropriate portions of the tails 63, 64 being 
displayed in the colour for a dissolve, for example ma- 
30 genta, and hidden frames not used in the dissolve are 
indicated by the appropriate portions of the tails 63, 
64 being displayed in the colour for a splice, for ex- 
ample yellow. Thus, in the above example the last 
five hidden frames in the first clip would be represent- 
35 ed by a magenta tail portion and the other five frames 
would be represented by a yellow tail portion. Similar- 
ly, the first hidden frame of the second clip would be 
represented by the display of a yellow tail portion and 
the other five hidden frames would be represented by 
40 a magenta tail portion. 

Once a dissolve has been defined on the display 
screen, it can be previewed by running the combined 
clip containing the dissolve at video rate. A further 
control icon 34e to 36e is displayed beneath respec- 
45 tive reel bars 34 to 36 and enables the user to enter 
the preview mode. When the viewing control icon 34e 
to 36e is selected by use of the cursor the system re- 
sponds by displaying at full size on the monitor the 
frame displayed nearest to the centre of the working 
so space when the viewing icon was selected. A menu 
of functions (such as play, fast, forward, fast reverse, 
stop, etc) is also displayed on the monitor and the se- 
lected clip can be previewed by the use by selection 
of the desired functions with the cursor. In this pre- 
ss view mode of operation frames comprising the first 
clip are read in order from one path eg 15 of the video 
store 11 and passed via the video processor 27 to the 
display store 22 for display on the monitor 25. During 
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the dissolve frames comprising the second clip are si- 
multaneously read in order from the video store 11 via 
the other path 16 and are passed together with re- 
spective frames from the first clip to the processor 27. 
Respective frames from the first and second clip are 
combined in a weighted combination by the video 
processor 27 and the resulting combined frames are 
passed in order to the display store 22 for display of 
the combined image on the monitor 25. 

The above discussed previewing of a dissolve is 
made possible by the provision of a video store 11 
with two bidirectional paths 15, 16 each having a 
bandwidth sufficient to transmit video data at video 
display rates. Again, it should be noted that during 
preview there is no alteration to the video data stored 
in the video store 11 and this frees the user to experi- 
ment with different edits before making a commitment 
to a particular combination of clips. 

The system 1 0 also includes the facility to key to- 
gether simultaneously several clips on a frame by 
frame basis. In the exemplary system described here- 
in this keying facility is selected by moving the reel 
bar 35 to the right in Figure 4 thereby widening the 
working area 31 defined between reel bars 34 and 35. 
Video clips 37, 38, 39, 40 are each selected by use of 
the cursor and by manipulation of the stylus on the 
touch tablet as previously described. Selected clips 
are each brought into the working area 31 by further 
movement of the stylus and each new clip may be 
placed in front, behind, or between clips already dis- 
played in the working area 31. The clips 37 to 40 to- 
gether form a stack and the position of each clip 37 
to 40 determines the rank or priority of the clip in the 
stack. Each clip 37 to 40 can be manipulated indepen- 
dently or in combination with other clips in the stack. 
Thus, for example, clip 38 can be taken out of the 
stack by attaching the cursor 57 to the clip and mov- 
ing it to working area 32. The clip can then be repli- 
cated or otherwise modified before being returned to 
its original position or another position in the stack. A 
clip can also be scrolled independently of other clips 
in the stack. Clips are connected to the cursor and 
moved sideways in the working area 31 until, for ex- 
ample, clip 38 does not overlap and is not overlapped 
by any of the other clips. Clip 38 can then be scrolled 
independently, as described hereinabove. 

The system is arranged such that a group of co- 
incident frames, corresponding ones from each of the 
clips in the stack, is available in the display store 22 
so that the composite result can be displayed on the 
monitor. This enables the user to preview the result 
before the clips are finally combined and committed 
to the video store 11, and to make corrections to in- 
dividual clips so that the composite clip contains the 
desire result. For example, the colour correction in 
one clip can be adjusted so as to match the coloration 
in another of the clips while both clips or frames from 
both clips are displayed simultaneously. If desired 



one foreground clip, eg clip 37, can be viewed against 
the background clip, eg clip 40. 

The video processor 27 includes a keyer circuit 
(not shown) by which a blue screen key can be gen- 

5 era ted. Such a keyer circuit is disclosed in our US Pa- 
tent No. 4,667,221, the teachings of which are incor- 
porated herein by reference. A user defined key or 
stencil may also be created by the graphics processor 
29 on a frame-by-frame basis for some or all frames 

10 in a selected clip. Previously defined key clips can be 
associated with respective video clips. The display 
store 22 is arranged to add data representing a black 
back-shadow to video clip data for which there is an 
associated key clip as the clip data is written to the 

15 display store 22. The black backs had ow (not shown) 
facilitates identification of a keyed clip when frames 
of the clip are displayed on the monitor 25. 

When a stack is defined in the working area 31 
the key clips associated with the respective video 

20 clips are assigned a priority or rank such that when 
the clips 37 to 40 are combined, keyed features in clip 
40 are seen in the foreground obscuring features in 
the other clips, whereas features in the clip 37 form 
the background and are obscured by keyed features 

25 in the other dips 38 to 40. 

It should be noted that data relating to any clip 
displayed on the monitor can be transferred to the ef- 
fects processor 28 and/or the graphics processor 29 
for creation of a key, or indeed for modification by any 

30 of the other operations provided by the two proces- 
sors 28, 29. 

Once the user is satisfied with the result of his 
editing, whether it be a simple cut and insert, a dis- 
solve or a layered sequence of stacked clips with col- 

35 our and texture changes to some of the dips therein, 
the resulting clip can be committed to memory. Clip 
data is read from the video store 11 and combined 
frame by frame by the processors 27, 28 and 29 as 
appropriate before being written back to the video 

40 store 11. 

In the case of a simple cut and insert splice edit 
there is no combining as such to be performed by the 
processors 27 to 29 and so the operation is simply 
one of re-indexing the frame data held in the store 11 . 

45 In the case of a dissolve edit it is necessary to com- 
bine two frames from different dips and so data for 
the two dips is read a frame at a time from the store 
11 along one of the paths, eg path 15, and the result- 
ing combined frame data is written back to the store 

so along the other of the paths, eg path 16. It will be ap- 
preciated that since each path only has sufficient 
bandwidth for one video channel the reading of 
frames for both dips via only path 15 will effectively 
halve the rate at which the two dips are read from the 

55 store 11 . However, this halving of the reading rate is 
unimportant since the combining operation corre- 
sponds to a final commitment by the user and can be 
performed without being displayed on the monitor 25. 
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There wilt be a similar or greater reduction in speed 
when plural clips of a stack as abovedescribed are 
combined but again this reduction is unimportant to 
the working of the system. 

Having thus described the present invention by 
reference to a preferred embodiment it is to be well 
understood that the embodiment in question is exem- 
plary only and that modifications and variations such 
as will occur to those possessed of appropriate knowl- 
edge and skills may be made without departure from 
the spirit and scope of the invention as defined in the 
appended claims and equivalents thereof. 

For example, the four relatively high capacity disc 
drives in the video store could be replaced by a large 
number of smaller disc drives. An important consider- 
ation in the construction of the video store is that it 
should have sufficient bandwidth for at least two vid- 
eo clips, and more than two if so desired, to be trans- 
ferred each at video rate. By increasing the number 
of disc drives comprising the store 11 the complexity 
of transferring data will increase also. Random ac- 
cess to the frames stored in the discs can neverthe- 
less be maintained by enlarging the cache stores in 
the buffers and/or the compressor, the size of the 
cache stores depending on the excess bandwidth 
and seek time of the discs, arid on the number of clips 
to be transferred simultaneously at video rate. The 
video store may be used in a high definition television 
system simply by increasing the number of discs used 
and if necessary enlarging the cache stores. 



Claims 

1. An electronic video store comprising storing 
means for storing digital data representing multi- 
ple video frames and access means for providing 
random access input and output of video frames 
to and from the storing means and including at 
least two ports connected to the storing means 
and each capable of transferring data to or from 
the storing means relating to a respective video 
frame at video rate. 

2. A store as claimed in claim 1, wherein said storing 
means comprises at least two separate storing 
units each connected to the said two ports and 
data representing each video frame is distributed 
between the storing units. 

3. A store as claimed in claim 2, wherein each stor- 
ing unit comprises at least two disc drives which 
together are capable of transferring video data at 
video rate. 

4. A store as claimed in claim 3, wherein each frame 
comprises two interleaved fields, and the access 
means is arranged for transfer of data stored or 



to be stored in the storing units and arranged 
therein such that each video frame data repre- 
senting one field is stored in one storing unit and 
data representing the other field is stored in the 
5 other storing unit. 

5. A store as claimed in claim 4, wherein the access 
means is arranged for transfer of data to or from 
each said storing unit, said data being arranged 

10 such that each field is divided into chrominance 
and luminance data groups which are stored re- 
spectively in the disc drive of the storing unit 

6. A store as claimed in any of cia ims 2 to 5, wherein 
is the access means comprises two buffers respec- 
tively associated with the two storing units for 
buffering data in transfer between each of the 
storing units and the two ports. 

20 7. A store as claimed in claim 6, wherein the access 
means further comprises an interface between 
the buffers and the two ports, which interface 
converts video data between a video format at 
said ports and a storing format at said buffers. 

25 

8. A store as claimed in claim 7, wherein the inter- 
face comprises a cache store for accumulating a 
multiplicity of frames of data to facilitate the trans- 
fer of data between the two ports and the storing 

30 units. 

9. Astore as claimed in claim 6 or 7 or 8 t wherein the 
two buffers each comprise a cache store for ac- 
cumulating a multiplicity of frames of data to fa- 

35 cil itate t he transfer of data between t he two ports 
and the storing units. 

10. An electronic video processing system for com- 
bining a plurality of video clips, the system com- 

40 prising storing means for storing a multiplicity of 
video frames and access means for providing 
random access transfer to and from the storing 
means of said video frames comprising said plu- 
ral video clips and including at least two ports 

45 connected to the storing means and each capa- 
ble of transferring data to or from the storing 
means at video rate, the system further compris- 
ing a processor responsive to user operable input 
means for processing data from the storing 

so means representing user selected frames of said 

two video cKps to produce a combined video clip, 
and a monitor for simultaneously displaying at 
least some of said multiplicity video frames. 

55 11. A system as claimed in claim 10, further compris- 
ing a display store for storing image data repre- 
senting an image to be displayed on the monitor, 
the processor being operable in an editing mode 
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for controlling the writing of data representing plu- 
ral selected frames to the display store for display 
of said plural selected frames on the monitor. 

12. A system as claimed in claim 11, wherein the 
processor is arranged to apply user selected ef- 
fect to user selected frames in the video clips, the 
effects and frames being selectable by manipula- 
tion of said user operable input means. 



15. A system as claimed in any of claims 11 to 14, 
wherein the processor is arranged to generate 
cursor data representing a cursor in response to 
user manipulation of said user operable input 
means, said processor being arranged to write 
the cursor data to the display store for display of 
said cursor on the monitor. 

16. A system as claimed in claim 15, wherein the 
processor is arranged to respond to the action of 
the cursor being disposed by manipulation of the 
user operable means at a position between two 
adjacent displayed frames of a single clip and 
moved rapidly away from said position, the proc- 
essor responding by modifying the data in the 
display store so as to divide the single clip into 
two new clips, one of the new clips including one 
and the other new clip including the other of said 
adjacent displayed frames. 

17. A system as claimed in claim 15 or 16, wherein in 
order to effect dividing of said single dip into two 
new clips the processor responds to the dis- 
played cursor being moved rapidly in a substan- 
tially horizontal direction by corresponding ma- 
nipulation of the user operable input means. 

18. A system as claimed in claim 15 or 16 or 17, 
wherein the processor is arranged to respond to 
manipulation of said user operable input means 
by modifying the data in the display store so as 



to cause selected displayed frames relating to 
one clip to become attached to the displayed cur- 
sor, and to further manipulation of the input 
means by modifying the data in the display store 
5 to cause both the cursor and the selected dis- 

played frames to move on the display. 

19. A system as claimed in claim 18, wherein the 
processor is arranged to respond to the position- 
ing of the selected frames attached to the cursor 
at other displayed frames relating to another clip 
by modifying the data in the display store so as 
to combine frames of said one clip associated 
with said selected frames with frames of said an- 
other dip associated with said displayed frames 
to create a combined dip. 

20. Asystem as claimed in claim 1 9 wherein the proc- 
essor is arranged such that when said selected 
frames are overlaid on the display monitor on 
said displayed frames, data in the display store is 
modified so that a marker is displayed with the 
combined dip for indicating that the combined 
clip has a number of hidden selected frames as- 
sociates therewith. 

21. A system as daimed in daim 19 or 20, wherein 
the processor is arranged to respond to the over- 
laying of frames attached to the cursor at said 

30 other displayed frames by modifying the data in 
the display store so as to replace the frames of 
said one dip associated with the overlaid dis- 
played frames with frames of said another clip as- 
sociated with overlaying frames. 

35 

22. A system as daimed in claim 19 or 20, wherein 
the processor is arranged to respond to the over- 
laying of frames attached to the cursor at said 
other displayed frames by modifying the data in 

40 the display store so as to combine the overlaid 
frames and the overlaying frames on a frame by 
frame basis, thereby creating a combined dip 
which includes a dissolve between the dips form- 
ing the combined dip. 

45 

23. A system as daimed in daim 21, wherein the 
marker is displayed in a first colour to indicate 
that in the combined clip the overlaid frames are 
replaced with the overlaying frames. 

50 

24. A system as daimed in daim 22, wherein the 
marker is displayed in a second colour to indicate 
that the combined clip is created by dissolving 
between overlaid and overlaying frames. 

55 

25. A system as daimed in claim 22 or 24, wherein 
the total number of frames used in dissolving be- 
tween the one dip and the other clip forming the 



13. A system as daimed in daim 12, wherein the 
processor is arranged to write data to the display 
store such that when the image represented by 
the data stored in the display store is displayed, 
the monitor is divided into a plurality of working is 
areas, said plural selected frames being dis- 
played at a reduced size in one or more of said 
working areas. 

14. A system as daimed in claim 13, wherein the 20 
processor is arranged to write data to the display 
store such that when the image represented 
thereby is displayed, some at least of the working 
areas each include one or more frames at a re- 
duced size, the frames corresponding to a por- 25 
tion of one or more of video dips. 
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dissolve is selectable, the marker comprising a 
central portion displayed in one colour indicating 
frames used in the dissolve and end portions dis- 
played in another colour indicating the frames in 
the one clip and in the other clip not used in the 5 
dissolve. 

26. A system as claimed in any of claims 18 to 25, 
wherein once a clip has been identified by manip- 
ulation of the input means the processor is ar- 10 
ranged to respond to the cursor being moved off 

and then back on to the display in response to 
manipulation of the input means by modifying the 
data in the display store so as to attach a copy of 
the identified clip to the cursor. is 

27. A system as claimed in any of claims 13 to 26, 
wherein one of said working areas is defined as 
a plural clip working area in which a plurality of 
clips can be processed concurrently. 20 

28. A system as claimed in claim 27, wherein at least 
some of the. plurality of dips in the plural clip 
working area have an associated control image 

clip, the processor being arranged to enable the 25 
displayed plural clips to be placed in a stack and 
to assign a priority to each clip depending on the 
position of the clip in the stack. 

29. A system as. claimed any of in claims 16 to 28, 30 
wherein the processor is operable in a preview 
mode in which data from the random access vid- 
eo frame store representing the plurality of video 
clips is combined in accordance with the modifi- 
cations made in said editing mode to the data in 35 
the display store to produce data representing a 
preview clip which is output to the monitor for dis- 
play. 

30. A system as claimed in claim 29, wherein the pre- 40 
view clip data is output for display of the clip at 
video rate. 

31. A system as claimed in any of claims 11 to 30, 
wherein the processor is operable in a committing 45 
mode in which data from the random access vid- 
eo frame store representing the plurality of video 
clips is combined in accordance with the modifi- 
cations made in said editing mode to the data in 

the display store to produce data representing a 50 
final combined clip which is written back to the 
random access video frame store, the processor 
thereby committing the final combined clip to 
memory. 
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The system comprises a store 11 having sev- 
eral disc drives for storing multiple video frames 
in random access order. The store 1 1 also has at 
least two* ports 15, 16 each capable of transfer- 
ring data relating to a respective video clip at 
video rate. The system further comprises pro- 
cessors 27, 28 for processing frame data as- 
sociated with at least two video clips and a 
monitor for displaying selected ones of said 
multiple video frames. In a preview mode of 
operation video clips can be read from the store 
11, combined and displayed on the monitor at 
video rate, and changes can be made to the 
dips and the manner in which they are com- 
bined without committing the combination 
the store. 



^ < 



to 



3^ 



O 



□ 



-o 



/ 



\ 



15 



Our 
aicr 

QO 



... \l 



Jouve, 18, rue Saint-Denis, 75001 PARIS 



EP 0 560 624 A3 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 93 30 1889 



DOCUMENTS CONSIDERED TO BE RELEVANT 



i Category 



Citation of document with indication, where appropriate, 
of relevant passages 



Relevant 

to claim 



CLASSIFICATION OF THE 
APPLICATION (Int.C1.5) 



D, A 
A 

D,A 



WESCON 86/C0NFERENCE RECORD 

vol. 30 , November 1986 , LOS ANGELES » CA, 

USA 

pages 1-11 

S. RAJPAL ET AL. 'New features on static 
RAMs and dual-port RAMs enhance system 
performance 1 

* page 1, left column, line 10 - line 34 * 

* page 4, right column, line 1 - page 5, 
right column, 1 ine 16 * 

GB-A-2 156 627 (QUANTEL LIMITED) 

* the whole document * 

GB-A-2 179 819 (QUANTEL LIMITED) 

* the whole document * 

EP-A-0 171 829 (QUANTEL LIMITED) 

* page 4, line 8 - page 6, line 7 * 

EP-A-0 396 415 (QUANTEL LIMITED) 

* column 2, line 26 - column 3, line 15 * 

GB-A-2 248 362 (RANK CINTEL LIMITED) 
page 1, line 18 - line 33 * 
page 6, line 19 - page 7, line 14 * 



The present search report has been drawn up for ail claims 



1,8-10 



H04N5/262 



1-5,10 

1,10 

1,10 

10,15 

1,4,10 



TECHNICAL FIELDS 
SEARCHED (lat.a.5) 



H04N 
G09G 
G11C 



Place •dtartl 



THE HAGUE 



' D*U *t eoapUttoa of tta fctarck 

5 November 1993 



DE DIEULEVEULT, A 



CATEGORY OK CITED DOCUMENTS 

X : particularly relevant if taken alooe 

Y : particularly relevant if corabioed with another 

document of the same category 
A : technological background 
O : noo-written disclosure 
P : intermediate document 



T : theory or principle troderlying toe invention 
E : earlier patent document, but published on, 01 

after the filing date 
D : document dted la tie application 
L : document cited for otber reasons 



A : member of the sane patent family, corresponding 
document 



2 



